Ensemble Learning with Dynamic Ordered Pruning for Regression

A novel method of introducing diversity into ensemble learning predictors for regression problems is presented. The proposed method prunes the ensemble while simultaneously training, as part of the same learning process. Here not all members of the ensemble are trained, but selectively trained, resulting in a diverse selection of ensemble members that have strengths in different parts of the training set. The result is that the prediction accuracy and generalization ability of the trained ensemble is enhanced. Pruning heuristics attempt to combine accurate yet complementary members; therefore this method enhances the performance by dynamically modifying the pruned aggregation through distributing the ensemble member selection over the entire dataset. A comparison is drawn with Negative Correlation Learning and a static ensemble pruning approach used in regression to highlight the performance improvement yielded by the dynamic method. Experimental comparison is made using Multiple Layer Perceptron predictors on benchmark datasets.